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SUBSTITUTE SPECIFICATION WITH MARKINGS 

METHOD AND SYSTEM FOR MEASURING DEGRADATIONS OF A VIDEO IMAGE 
INTRODUCED BY CODING WITH REDUCTION IN THROUGHPUT 

BACKGROUND OF THE INVENTION 

(1) Field of the Invention 

[0001] This invention relates to a method and system for 
measuring degradations of a video image introduced by a coding 
system with reduction in throughput. 

[0002] It is particularly but not exclusively applicable to 
the domain of low throughput or very low throughput digital 
audiovisual signal distribution networks, and to the domain of 
production of such signals. It is particularly applicable to 
surveillance of the service quality of a digital audiovisual 
signal broadcast network. 

(2) Prior Art 

[0003] Digitizing of video signals provides a means for 
copying, storing and transmitting this type of information 
while maintaining a constant image quality. However, in 
practice, the large quantity of information transferred by 
video images requires the use of digital compression methods 
to reduce the binary throughput . 

[0004] A compression method that is very widely used in video 
is described in standard ISO/CEI 13918 MPEG2 . This algorithm 
is said to be of the "with losses" type since the restored 
image after decoding is not identical to the original. This 
algorithm is based on a division of the image into blocks and 
application of a transform, for example of the discrete cosine 
transform type, to the pixels in each block to obtain a 
frequency representation of the luminance amplitude of pixels 
in the form of one coefficient for each pixel in the block. 
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[0005] In order to maintain an acceptable quality for the 
final television viewer, the compression algorithms take into 
account perception properties of the human vision system. 
However, throughput constraints imposed by transmission 
systems require the application of compression ratios that 
have an influence on the image quality perceived by the 
television viewer. 

[0006] It is found that the importance of degradations caused 
by coding depends both on the compression ratio and the 
complexity of images. These degradations are particularly 
important when the image is more complex, and particularly 
related to movement of objects, brightness and texture. 

[0007] The degradations that appear in the images following 
application of the MPEG2 coding technique include granular 
errors, deformations of contours, the appearance of so-called 
"exotic" contours and block effects. 

[0008] Therefore it would appear necessary to continuously 
evaluate the quality of broadcast images. There are widely 
used subjective evaluation methods for this purpose, that make 
use of human evaluation. However, these methods are difficult 
to use and cannot be used on a broadcasting network in 
operation in real time. 

[0009] There are other so-called "with reference" methods 
based on comparison of the image for which the quality is to 
be evaluated with a reference image. The reference image is 
usually an image that corresponds to the image to be analyzed 
before it is coded and / or transmitted. This solution is not 
very practical, because it requires access to one or several 
reference images. Furthermore if the video image is 
transmitted, there is also the problem of transporting the 
reference image to the place at which the image to be analyzed 
is received. 

[0010] Other so-called "without reference" solutions are used 
to automatically analyze images without needing to make a 
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comparison with reference images. The efficiency and 
robustness of each of these solutions lies in the method used 
to measure parameters related to image quality. 

[0011] Some of these solutions are based on detection of the 
block effect made in the spatial domain by gradient 
calculations at block boundaries. To avoid confusion between 
the boundary of an object in the image with a block effect, 
the gradient is compared with intra-block gradients. The block 
effect is detected by means of a decision criterion applied to 
the behaviour of inter block and intra block gradients. 

[0012] Thus, the Rohde & Schwarz Company has developed a block 
effect detection method consisting of calculating a horizontal 
gradient vector for each macro block in the image, and 
calculating an average of each component of the vector over 
the entire image. Variations of components of this vector over 
time are used to bring out components with marginal behaviour 
that represent block boundaries degraded by the compression 
processing transform. Detection of these marginal components 
provides a means for determining a block effect detection 
criterion representative of the image degradation. 

[0013] This principle for calculating the gradient is also 
described in patent FR 2 805 429 filed by the Applicant. This 
patent application describes a method based on the combination 
of a binary gradient image and a movement "pseudo vectors" 
image calculated from at least two successive images. A 
combination of these two images provides a means for 
estimating a ratio of false contours in the image, and is then 
used to evaluate a quality mark. 

[0014] In patent FR 2 785 116 filed by the applicant, the 
gradients calculated on the entire image to be analyzed are 
passed through psychovisual filters that translate the 
contextual masking effect. A ratio of boundaries of detected 
visible blocks is then calculated searching for a pseudo 
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periodicity among high value gradients, the image quality 
being evaluated based on this ratio. 

[0015] It is found that methods based on the calculation of 
gradients apply filters to only consider a certain type of 
image contents: boundaries or high frequencies. Therefore 
these methods can only be used to analyze part of information 
contained in the image. The result is that they have limited 
reliability in terms of detection of image degradations. 
Furthermore, methods based on the use of gradients to estimate 
the boundary or for the detection of contours on the image are 
relatively sensitive to noise, which affects the reliability 
of the estimate of the quality of the intrinsic content of the 
image . 

[0016] Furthermore, methods based on the calculation of the 
average on the entire image drastically reduce the importance 
of degradations located in a part of the image, which makes it 
difficult to detect such local degradations and therefore 
affects the reliability with which the image quality is 
evaluated . 

[0017] Some methods allow an analysis on several successive 
images, to reduce these disadvantages. Therefore these methods 
cannot be used to analyze an isolated image outside the scope 
of the video. 

SUMMARY OF THE INVENTION 
[0018] The purpose of the present invention is to eliminate 
these disadvantages. This purpose is achieved by providing a 
method for measuring degradations of a digitized image, 
introduced when coding the image, the method consisting of 
dividing the image into coding blocks using a coding grid and 
applying a coding processing on pixel data in each block, 
making use of a block transform calculation and an inverse 
block transform calculation. According to the invention, this 
process includes steps of: 
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[0019] - determining the coding grid of the coded image, in 
order to find the image division into coding blocks, used when 
coding the image, 

[0020] - shifting the coding grid with respect to the coded 
image, so as to define an image division into analysis blocks 
each covering a boundary between two adjacent coding blocks, 

[0021] - applying the block transform calculation to pixel 
data in the coded image using the shifted coding grid to 
obtain transformed coefficients for each analysis block 
defined by the shifted coding grid, 

[0022] - extracting coefficients that could be affected by a 
block effect resulting from coding of the image, from the 
transformed coefficients, 

[0023] - applying the inverse block transform calculation to 
the extracted transformed coefficients to determine the pixel 
data for each analysis block, 

[0024] - for each analysis block, estimating an indicator of 
the degradation due to block effects, using pixel data in the 
coded image and pixel data in each analysis block, obtained by 
the inverse transform calculation, and 

[002 5] - determining an image degradation measurement by 
summing the degradation indicators of each analysis block . 

[0026] According to a special feature of the invention, the 
estimation of a degradation indicator for each analysis block 
comprises steps of: 

[0027] - calculating an average of inter pixel differences at 
the inter block boundary of the coding grid, covered by the 
analysis block, using pixel data obtained for the analysis 
block, 

[0028] - calculating an average and a standard deviation 
applicable to pixels in the two adjacent blocks on the coding 
grid, partially covered by the analysis block, 
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[0029] - calculating a weighting factor as a function of the 
average and the standard deviation obtained for the analysis 
block, and 

[0030] - calculating a spatial activity of the analysis block 
using spatial activities determined for each of the two 
adjacent blocks in the coding grid partially covered by the 
analysis block, 

[0031] - the analysis block degradation indicator being 
determined as a function of the calculated average of inter 
pixel differences, the weighting factor and the spatial 
activity of the block. 

[0032] Advantageously, the analysis block degradation 
indicator is obtained using the following formula: 



inter block boundary of the coding grid covered by the 
analysis block, w i( j is the weighting factor, y is a predefined 
constant, and ACT^j is the spatial activity of the analysis 
block. 

[0033] According to another special feature of the invention, 
the transform calculation is applied to coding blocks of the 
coded image, the spatial activities determined for each of the 
two coding blocks being obtained from the transformed 
coefficients for each of the two coding blocks. 
[0034] Advantageously, the spatial activities determined for 
each of the two coding blocks are obtained from the following 
formulas : 




l + ^ACTy 



in which Ali,j is the average of inter pixel differences at the 
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in which DCy and ACy (u,v) and DC-j and ACjj(u,v) are the 



transformed coefficients for each of the two adjacent coding 
blocks partially covered by the analysis block, and Nill(u,v) 
is a masking function modelling masking by neighbourhood. 
[0035] Preferably, the average and standard deviation 
calculated for each analysis block are determined from 
transformed coefficients for each of the two adjacent coding 
blocks partially covered by the analysis block. 

[0036] According to another special feature of the invention, 
the weighting factor is obtained by the following formula: 



/x i#j and <Ti,j being the average and standard deviation 
respectively calculated for each analysis block and £ being a 
parameter corresponding to the maximum sensitivity of the 
human eye . 

[0037] According to another special feature of the invention, 
analysis blocks that could contain a block effect are selected 
before estimating a degradation indicator for each analysis 
block . 

[0038] Advantageously, the prior selection comprises a step 
consisting of separating analysis blocks for which the 
extracted transformed coefficients are greater than a 
predetermined threshold . 

[0039] Preferably, the prior selection comprises a step 
consisting of selecting analysis blocks with pixels with an 



Wi,j (/ii,j,ai #j/ <;) = < 




in which X = 
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energy representing a significant proportion of the energy of 
the block, at the inter block boundary of the coding grid 
covered by the analysis block. 

[0040] According to another special feature of the invention, 
the coding grid is shifted horizontally with respect to the 
coded image . 

[0041] Alternately, the coding grid is shifted vertically with 
respect to the coded image. 

[0042] Preferably, the block transform calculation is a 
discrete cosine transform calculation. 

[0043] The invention also relates to a system for measuring 
degradations of a digitized image introduced when coding the 
image, the system comprising calculation means for 
implementing the method defined above. 



BRIEF DESCRIPTION OF THE DRAWINGS 
[0044] One preferred embodiment of the invention will be 
described below as a non-limitative example with reference to 
the appended figures, wherein: 

[0045] Figure 1 diagrammatically represents the measurement 
system according to the invention, integrated into an image 
processing system; 

[0046] Figure 2 shows the measurement system represented in 
Figure 1 in more detail; 

[0047] Figure 3 shows two image pixel blocks to illustrate the 
method according to the invention, 

[0048] Figure 4 shows part of the system shown in Figure 2 in 
more detail; 

[0049] Figure 5 shows a curve representing a weighting 
function used by the method according to the invention; 

[0050] Figure 6 shows other curves illustrating another 
weighting function used by the method according to the 
invention; 



8 



05-300 



[0051] Figure 7 shows curves illustrating the variation in 
degradation measurements obtained using the method according 
to the invention applied to a sequence of images. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT ( S ) 
[0052] Figure 1 represents a degradation measurement system 1 
according to the invention that is designed for use as a probe 
that could be applied to any point in a digital image 
processing system such as broadcasting system or a video 
signal production system. 

[0053] The measurement system 1 is designed particularly to 
measure the quality of images output from a coding system 10 
with reduced throughput respecting the MPEG 2 standard. For 
this purpose, it is based on a frequency and time filtering 
combination . 

[0054] This system is applicable whenever there is a need to 
identify coding defects in a digital video signal, 
particularly to determine the appropriate throughput for a 
given image sequence as a function of the expected quality. 

[0055] The coding principle according to the MPEG 2 standard 
consists of dividing the digitized image into blocks of N x N 
pixels (for example where N is equal to 8) using a coding 
grid, and applying a discrete cosine transform (DCT) to each 
block to change from the spatial domain to the frequency 
domain, and then to cancel some components of the transform 
corresponding to high frequencies before applying the inverse 
DCT transformation to change back to the spatial domain, in 
other words so as to recover the pixels from the corresponding 
block . 

[0056] The discrete cosine transform processing consists of 
making a calculation of transform coefficients AC and DC 
obtained from the following formulas, to each block Bi,j of 
N x N and pixels in the image: 
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and 



in which 



2 N-1N-1 

DC i#j = F i , j (0 / 0)=— JJfy(x,y) (1) 

w x =0y=0 



ACi,j(u,v) = Pi # j(u # v) f where u + v * 0 (2) 



Pi.j(u,v)-— ZZ f i,j( X >y) C °S 7IU COS TIV^-^JJ, (3) 



N 2 x= 0y=0 



V 



2N 



V 




COS 71V — \\ (4) 



fi,j(x,y) represents the luminance of the pixel at point (x,y) 
in block B if j, x and y being the horizontal position index and 
the vertical position index respectively of the pixel in the 
block Bi,j, and u and v are between 1 and N-l and represent the 
indexes of the horizontal and vertical spatial frequencies 

respectively, and c(0) = ^= and c (u) = 1 if u * 0. 

V2 

[0057] The inverse processing of the discrete cosine transform 
consists of applying the following formula to the coefficients 
DCi,j and AC i# j(u,v) of each block B ifj : 

t 4 Vv 1 , w « / x ( f 2x + ^ 

W x >y) = ^Z X c ( u ) c ( v ) F ij(^ v ) cos ™ -z^r . 

N u=0v=0 V V 2N J) 

[0058] In Figure 2, the measurement system 1 comprises a 
synchronisation module 11 that synchronises each coded image 
to be processed with its coding grid, in other words the 
module 11 divides the image into blocks of N x N pixels. One 
example of a synchronisation processing that could be done by 
this module is described in patent application FR 2 769 452 
filed by the Applicant. This synchronisation processing 
identifies the division into blocks used by the previous 
coding processing done by the system 10, within the image, and 
therefore determines the coding block Bi,j to which each pixel 
in the image belongs. 

[0059] The coded image and its division into blocks are 
applied to two processing branches 2, 3, the first branch 2 
comprising a discrete cosine transform module 16, and the 
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second branch 3 comprising firstly a shift module 12 producing 
a horizontal shift of the coding grid by half a block with 
respect to the image. Figure 3 illustrates this horizontal 
shift by half a block, by showing a block Bi,j of 8 x 8 pixels 
horizontally shifted by half a block with respect to the 
coding grid illustrated by the adjacent blocks 21 and 22. 
[0060] In the second processing branch 3, the coded image and 
the shifted decoding grid at the output from the shift module 
12 are applied to the input of a discrete cosine transform 
module 13. A horizontal frequencies extraction module 14 then 
processes transform coefficients output from module 13. In 
fact, this module consists of assuring that the frequency 
domain only retains frequencies (coefficients AC i# j(u,v) and 
DCi,j) that could make a vertical block effect visible. If 
N = 8, it could thus be estimated that it is only necessary to 
retain coefficients DC± t j, ACi,j (u,v) such that: 

0<u<8ifv=0, 

2<u<8ifv=l, and 

6<u<8ifv=2 

[0061] The retained coefficients ACi,j(u,v) and DCi,j are then 
processed by an inverse DCT transformation module 15 to 
retrieve spatial information Ii,j(x,y) = I(8i+x, 8j+y) related 
to the retained frequencies obtained using formula (4) . In 
this respect, it is possible to consider only the fourth and 
fifth columns 24, 25 concerned by the block effect due to 
coding, and which correspond to the boundary 2 3 between the 
blocks 21 and 22 of the coding grid in Figure 3. 

[0062] Therefore, the output from branch 3 provides pixel data 
I(8i+x, 8j+3) and I (8i+x, 8j+4) for the complete image, where 
i and j are the indexes of shifted blocks of the image using 
the shifted coding grid obtained at the output from the shift 
module 12. The output from branch 2 provides coefficients DCi,j 
and ACi,j(u,v) for each image block related to the coding grid 

11 



05-300 



at the output from the synchronisation module 11. For each 
shifted block Bi,j, the coefficients DCg and AC? (u,v) in the 

left block 21 and the coefficients DC^ and ACy(u,v) of the 
right block 22 are available. 

[0063] The data output from the two processing branches 2 and 
3, in other words at the output from modules 16 and 15, are 
applied to the input of a block effect evaluation module 17. 
[0064] The data at the input to module 17 shown in Figure 4 
are applied to the following modules: 

[0065] - a module 31 for calculating the average of inter 
pixel differences at the vertical boundary 23 between each 
pair of adjacent blocks 21, 22 in the coded image; 
[0066] - a module 32 for calculating the average and the 
standard deviation of the luminance of each block; and 
[0067] - a local contrast calculation module 34. 
[0068] The module 31 applies the following formula to pixel 
data output from the second branch 3 , to evaluate the average 
of inter pixel differences on each shifted block Bi f j: 

i 7 

= -Z I(8i + k,8j + 4)-(8i + k,8j + 3)| 



8 k=o 



(5) 



[0069] The average thus calculated actually describes the 
behaviour of the gradient operator at the vertical boundary 23 
of two adjacent blocks 21, 22 output from the image coding 
grid. 

[0070] In module 32, an average and a standard deviation 

Gi t j of the luminance are estimated for each block B i# j on the 
two corresponding adjacent blocks 21, 22 in the coding grid. 
For example, the AC and DC coefficients of the coding blocks 
in the DCT domain output by the first branch 2 could be used 
according to the following formulas: 

Mi.j = i(DC°+DC°) (6) 

and 
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[I i[AcP j( u,v)f + lL E[ACe j( u,v)f 



u,v=0 
u + v*0 



u,v=0 
u + v*0 



(7) 



[0071] Obviously, the average n± t j and the standard deviation 
Gi,j could also be obtained directly from the pixels of blocks 
21, 22 corresponding to each block Bi,j using classical 
formulas for calculating the average and the standard 
deviation . 

[0072] The averages and standard deviations calculated by the 
module 32 are then applied to a module 33 for calculation of a 
weighting factor w if j , a^j, Q for each block B it j. The 

module 33 applies the weighting function defined by the 
following formula to calculate this weighting factor: 



w 



i-3 



( 



1 + - 



In 



(8) 



l + <7, 



else, 



••J y 



In 



where 



l + o\. 



In 



1 + 



1 + ^y; 



(9) 



and ^ is a parameter corresponding to the maximum sensitivity 
of the eye . 

[0073] This weighting function is described in the publication 
"A Generalized Block-Edge Impairment Metric for Video Coding" 
by H.R Wu and M. Yuen, IEEE Signal Processing Letters, Vol. 4, 
No. 11, November 1997. However, the weighting function 
described in this publication has a discontinuity at point /z^j 
= when the standard deviation a± t j is not zero. Therefore, 
this function is adapted in the method according to the 
invention to be continuous at this point. 
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[0074] Figure 5 illustrates this weighting function in the 
form of a curve 41 as a funct ion of the grey level or the 
average luminance of pixels in the block, in the case in 

which £ is chosen to be equal to 81, corresponding to the 

maximum sensitivity of the human eye, and where a is equal to 
20 . 

[0075] The local contrast calculation performed by module 34 
consists in estimating the spatial activity ACTj^j of each 
block B if j in the transform domain, starting from coefficients 
DCi,j and ACi^, the coefficients ACi,j being weighted by a 
masking function to model masking by the neighbourhood. The 
spatial activity ACT i#j of each block B irj is thus estimated by 
applying the following formula: 

ACT ±/j = i(ACT5+ACT5) (10) 



where ACT G : = 1 



" 1 + DCfj 



S[ACfj(u,v)Nill(u,v)f (11) 



u,v=0 
u + v*0 



and AClfj = l -— i;[ACPj(u f v)NiU(u,v)f (12) 

y u+v^o 



in which Nill(u,v) is the masking function that can for 
example be the function described in the document "A Visual 
Model Weighted Cosine Transform for Image Compression and 
Quality Assessment" by N. B. Nill, IEEE Transactions on 
Communications, Vol. COM- 33, No. 6, June 1985. 

[0076] In this document, the masking function is given by the 
following formula : 

Nill(u,v) = A(co)H(co) (13) 

where co= Vu 2 + v 2 represents the radial frequency expressed in 

cycles per degree (u and v e[0,N-l]), 



A(co) = J- + — 



7C 2 



In 



Inn /47T7r 2 co , 
+ J + 1 



V 
14 



a V a 2 



(14) 
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H(co) = (0.31 + 0.69co)e~°' 29G> (15) 
and a is commonly equal to 11.636 deg" 1 for processing of 
luminance on 8 bits. 

[0077] Figure 6 illustrates this weighting function by 
representing the curve 43 showing the variation of H(co) , the 
curve 44 showing the variation of A(oo) and the curve 45 
showing the variation of the function Nill (co) , as a function 
of the spatial frequency co . 

[0078] In formulas (11) and (12) , normalisation by the DC 
coefficient provides a means for considering the match to the 
local contrast in accordance with the Weber's law, and one is 
added to the denominator so that the function is continuous at 
zero . 

[0079] Finally, the results of calculations carried out by 
modules 31, 33 and 34 are applied to a module 3 5 for 
evaluation of the image degradation u due to the edge effect 
resulting from coding. This module carries out the following 
calculation to estimate an image degradation measurement: 

o = ]>>y (i6) 
u 

where u i#j = — ^ ? (17) 

l + V |ACTy| 

represents the impact of block effects on block Bi,j and \j/ is 
an experimentally determined constant that adjusts the 
importance assigned to the spatial activity compared with the 
weighted average of inter pixel differences. 

[0080] The measurement of the degradation u estimated by this 
module corresponds to an indicator of the visibility by the 
human eye of degradations due to the block effect. 

[0081] Figure 7 shows variations of this measurement in the 
shape of curves 46, 47, 48, as a function of the image number 
in a sequence of video images, for throughputs of 9, 6 and 4 
Mbits/s respectively. 
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[0082] These curves show that image degradations are amplified 
as the throughput is reduced. Therefore, the coarse coding by 
quantification of DCT coefficients results in the block 
effect. The part of the image sequence in which the 
degradation measurement is high corresponds to images for 
which the spatial and time contents change suddenly. 

[0083] Processing done by module 17 can be optimized by 
submitting input block data to a selection using a module for 
selection of blocks Bij that could contain a block effect. To 
achieve this, this module compares the energy of central 
columns 24 and 25 of each block Bij with energy of the block. 
The block is selected to estimate the degradation if the 
energy of columns 24 and 25 represents a significant 
proportion, for example more than 25%, of the total energy of 
the block. Before this selection is made, blocks for which the 
DCT coefficients selected by the extraction module 14 are 
below a predefined threshold, for example close to zero, can 
be eliminated. 

[0084] Note that the processing that has just been described 
is designed to detect a vertical block effect. Obviously, this 
processing could be modified in an easily understood manner to 
detect a horizontal block effect. All that is necessary in 
module 12 is to make a vertical shift of a half block, and 
then in the frequency extraction module 14 to select 
frequencies that could be affected by a horizontal block 
effect, and finally in module 15 to consider all of the 
reconstituted pixels and only retain pixels in the fourth and 
fifth lines of the shifted block corresponding to the 
horizontal boundary between the two adjacent non- shifted 
blocks corresponding to the shifted block. 

[0085] It would also be possible to perform vertical and 
horizontal block effect detection processings in parallel, and 
to use the results output by these two processings at the same 
time, for example in module 35. 
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[0086] After comparing objective measurements obtained by the 
method and the system according to the invention, with marks 
assigned during subjective tests by panels of representative 
persons, it is found that there is a strong correlation 
between objective measurements and subjective measurements, 
confirming that the method and system according to the 
invention are efficient. 
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